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SUMMARY  PAGE 


THE  PROBLEM 

To  determine  Whether,  and  to  what  extent,  supplementing 
visual  waterfall  displays  with  redundant  auditory  signals  (i.e., 
bimodai  displays)  enhances  detection  of  sonar  targets.  This 
study  was  designed  to  rule  out  several  potential  artifacts  which 
complicate  the  interpretation  of  previous  research  on  bimodai 
sonar  displays.  This  study  also  investigated  the  extent  to  which 
signal  uncertainty  and  the  degree  of  spatial  compatibility 
between  visual  and  auditory  signals  in  bimodai  displays  affects 
target  detection  performance. 


THE  FINDINGS 

The  signal-to-noise  ratio  (SNR)  at  which  targets  were  just 
barely  detectable  was  1.1  dB  lower  for  bimodai  displays  than  for 
single -modality  displays.  This  result  is  in  close  agreement  with 
most  previous  studies  of  bimodai  sonar  displays.  The  results 
also  show  that  the  deterioration  in  detection  performance  with 
increased  signal  uncertainty  is  considerably  less  for  auditory 
displays  than  for  visual  displays.  In  this  study,  increased 
spatial  compatibility  between  visual  and  auditory  signals  did  not 
facilitate  detection  on  bimodai  displays.  More  effective  methods 
making  auditory  and  visual  signals  spatially  compatible  are 
recommended  for  further  research. 


APPLICATION 


The  findings  confirm  that  bimodai  displays  enhance  target 
detection  performance  on  sonar  waterfall  displays,  and  suggest 
that  auditory  displays  offer  advantages  for  operational 
conditions  with  high  signal  uncertainty. 


ADMINISTRATIVE  INFORMATION 

This  investigation  was  conducted  under  Naval  Medical 
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ABSTRACT 


An  experiment  was  conducted  to  determine  whether  bimodal 
(auditory  plus  visual)  displays  enhance  operators'  basic  ability 
(perceptual  sensitivity)  to  detect  sonar  targets.  The 
possibility  that  operators'  decisions  about  how  to  respond  when 
uncertain  (response  criteria)  contributed  to  the  findings  was 
ruled  out  by  using  data  collection  and  analysis  procedures  based 
on  the  Theory  of  Signal  Detectability.  Also  the  detectability  of 
the  visual  and  auditory  signals  used  in  the  bimodal  display 
condition  were  carefully  equated.  This  ruled  out  the  possibility 
of  a  false  bimodal  effect  due  to  operator's  responding  on  the 
basis  of  the  more  detectable  of  the  two  signals  on  the  bimodal 
display.  This  study  also  examined  the  effects  of  signal 
uncertainty  and  the  degree  of  spatial  correlation  (compatibility) 
between  the  visual  and  auditory  signals  on  the  bimodal  display. 

It  was  expected  that  spatially  correlated  auditory  signals  would 
facilitate  detection  in  the  bimodal  condition  by  cluing  the 
operator  where  to  look  on  the  visual  display. 

Increased  signal  uncertainty  produced  an  increase  in  the 
average  signal-to-noise  ratio  (SNR)  at  which  the  signal  could 
first  be  detected.  The  increase  in  the  SNR  needed  for  detection 
was  significantly  greater  for  the  visual  display  (3.0  dB)  than 
for  the  auditory  display  (1.2  dB) .  On  the  bimodal  displays, 
signals  were  detected  at  significantly  lower  SNRs  than  on  single¬ 
modality  displays  (average  1.1  dB  difference).  Increased 
compatibility  between  the  visual  and  auditory  signals  did  not 
increase  the  advantage  of  bimodal  displays;  nor  did  the  advantage 
of  bimodal  displays  change  with  signal  uncertainty. 

The  findings  show  that  bimodal  displays  improve  signal 
detection  performance  in  sonar  systems.  They  rule  out  the 
possibility  that  the  advantage  of  bimodal  displays  is 
attributable  to  changes  in  operators'  response  criteria  and/or 
artifacts  caused  by  differences  in  the  detectability  of  the 
visual  and  auditory  signals.  They  also  suggest  that  auditory 
displays  offer  advantages  for  real-world  sonar  operations,  where 
signal  uncertainty  is  often  high.  The  method  used  to  make 
auditory  and  visual  signals  spatially  compatible  in  this  study 
was  not  successful.  More  effective  methods  for  creating  spatial 
compatibility  are  recommended  for  future  research. 


EFFECTS  OF  BIMODAL  DISPLAYS  ON  SONAR  TARGET  DETECTION 


The  earliest  sonar  systems  relied  exclusively  on  the 
auditory  modality  to  present  information  to  the  operator.  Over 
the  past  decade  or  two  there  has  been  a  shift  toward  use  of 
visual  displays  in  sonar,  although  auditory  displays  are  still 
regarded  as  an  important  component  in  submarine  sonars.  These 
changes  from  auditory  to  bimodal  and  visual  displays  have  been 
made  in  the  absence  of  satisfactory  data  on  the  effects  of  the 
various  display  types  on  operator  performance.  Previous  research 
on  bimodal  information  presentation  has  tended  to  fall  into  one 
of  two  categories:  basic  research  whose  applicability  to  sonar 
operations  is  uncertain,  or  applied  research  plagued  by 
methodological  problems. 

A  number  of  studies  in  the  first  category  have  reported  that 
detection  performance  is  improved  when  the  same  information  is 
presented  both  visually  and  aurally  (Brown  and  Hopkins,  1967; 
Buckner  and  Me  Grath,  1963;  Fidell,  1970;  Loveless,  Brebner,  and 
Hamilton,  1970) .  However,  the  tasks  and  stimuli  employed  in 
these  studies  differed  in  important  respects  from  those  used  in 
sonar  systems. 

One  of  the  earliest  applied  studies  employed  a  vigilance 
task  (Colquhoun,  1975) .  His  subjects  monitored  visual,  auditory, 
and  bimodal  displays  for  long  periods  of  time  for  a  small, 
infrequent  change  in  the  signal.  The  signal  consisted  of  an 
increase  in  the  intensity  of  one  of  four  pure  tones  mixed  with 
broadband  noise  in  the  auditory  modality,  and/or  an  increase  in 
brightness  of  one  of  four  rings  on  a  visual  display.  The 
percentage  of  signals  detected  was  much  greater  for  the  auditory 
display  than  for  the  visual  display,  and  the  bimodal  display 
produced  a  small  additional  improvement  over  the  auditory 
display.  However,  the  false  alarm  rate  was  also  greater  for  the 
bimodal  display  than  for  the  auditory  display.  Therefore,  the 
slight  superiority  of  the  bimodal  display  may  have  been  due  to  a 
relaxation  of  the  response  criterion,  rather  than  an  increase  in 
perceptual  sensitivity. 

The  problem  of  interpretation  due  to  possible  shifts  in  the 
response  criterion  is  also  present  in  subsequent  applied  studies. 
Kobus,  Russotti,  Schlichting,  Haskell,  Carpenter,  and  Wojtowicz 
(1986)  presented  simulations  of  sonar  target  emissions  generated 
by  the  U.  S.  Navy  Sonar  Operational  Trainer  in  a  background  of 
sea  noise.  The  target  was  presented  at  any  one  of  10  sectors 
which  the  subject  could  search  sequentially.  The  target  intensity 
was  increased  until  the  subject  reported  its  presence  in  the 
correct  sector.  Visual  signals  were  presented  on  a  waterfall- 
type  display  (which  displays  frequency  horizontally  and  time 
vertically) .  For  some  signals,  the  visual  display  produced  best 
detection;  for  others,  auditory  presentation  was  best.  The  signal 
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level  at  which  detection  occurred  in  the  bimodal  condition  was 
not  significantly  different  from  that  for  the  better  single 
modality. 

Other  applied  studies  using  the  same  type  of  visual  display 
have  reported  significant  advantages  of  bimodal  presentation 
(Luria  and  Jacobsen,  1986;  Lewandowski  and  Kobus,  in  press) .  In 
both  studies,  bimodal  presentation  produced  a  reduction  of  about 
1.0  dB  in  the  signal  level  required  for  detection.  The 
possibility  cannot  be  ruled  out  that  discrepancy  between  these 
studies  and  Kobus  et  al.  (1986)  in  the  bimodal  effect  is  due  to 
shifts  in  the  response  criterion.  It  would  be  of  interest  to 
conduct  further  research  using  realistic  sonar  displays  and 
tasks,  to  determine  if  changes  in  response  criterion  between 
display  conditions  can  account  for  these  results. 

Another  potential  problem  arises  if  the  signals  are  not 
carefully  equated  for  detectability  in  the  auditory  and  visual 
modalities.  In  the  bimodal  condition,  the  subject  may  respond  on 
the  basis  of  the  more  detectable  of  the  two  signals.  If  the 
signal  were  sometimes  more  detectable  in  the  visual  modality,  and 
sometimes  in  the  auditory,  then,  on  the  average,  the  signal  level 
required  for  detection  in  the  bimodal  condition  would  be 
spuriously  less  than  those  in  either  the  visual  or  auditory 
conditions.  That  is,  there  would  be  an  apparent  advantage  of 
bimodal  presentation,  even  though  the  subjects  were  responding  on 
the  basis  of  one  modality  in  the  bimodal  condition. 

The  primary  purpose  of  the  present  research  was  to  determine 
whether  bimodal  displays  enhance  an  operator's  perceptual 
sensitivity  in  detecting  sonar  targets,  relative  to  single¬ 
modality  displays.  The  method  of  the  present  study  differed  from 
that  of  previous  applied  work  in  two  important  respects.  First, 
the  possibility  that  changes  in  subjects'  response  criteria 
contributed  to  the  observed  differences  among  experimental 
conditions  was  ruled  out  by  using  data  collection  and  analysis 
procedures  based  on  the  Theory  of  Signal  Detectability  (TSD) 
(Green  and  Swets,  1966) .  Second,  the  possibility  of  obtaining  a 
spurious  bimodal  effect  was  minimized  by  carefully  equating  the 
detectability  of  the  visual  and  auditory  signals  used  in  the 
bimodal  condition. 

This  study  also  examined  the  effects  of  signal  uncertainty 
and  increased  spatial  compatibility  between  visual  and  auditory 
information  in  bimodal  displays.  In  the  high  compatibility 
condition,  the  auditory  stimulus  (noise  and  signal,  if  any)  was 
lateralized.  This  caused  stimuli  with  predominantly  low 
frequencies  to  be  heard  toward  the  left  side  of  the  head,  while 
those  with  predominantly  highs  were  heard  toward  the  right.  The 
position  in  which  the  auditory  signal  was  heard  corresponded 
roughly  to  its  position  on  the  visual  display,  which  presented 
the  stimulus  spectrum  horizontally. 
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Signal  uncertainty  and  display  compatibility  were  of 
interest  because  they  represent  variations  in  the  detection  task 
and  alternatives  for  bimodal  display  design  that  occur,  or  could 
be  used,  in  real  sonar  operations.  The  fact  that  signal 
uncertainty  produces  a  decrement  in  detection  performance  is  well 
documented  (Sperling  and  Dosher,  1986?  Swets,  1984) .  In  contrast, 
it  is  at  present  unknown  whether  increased  compatibility  between 
display  modalities  enhances  bimodal  detection  performance.  With 
the  high  compatibility,  bimodal  display,  the  auditory  signal  may 
cue  the  operator  where  to  look  on  the  visual  display.  It  was 
therefore  anticipated  that  greater  spatial  compatibility  between 
auditory  and  visual  displays  would  be  associated  with  a  larger 
bimodal  advantage.  It  was  also  of  some  practical  interest  to 
determine  whether  display  compatibility  and  signal  uncertainty 
interact,  i.e.,  whether  increased  display  compatibility  reduces 
the  performance  decrement  due  to  increased  signal  uncertainty. 

METHOD 

Overview;  Effects  of  three  major  variables  on  detection 
performance  were  investigated  in  a  simulated  sonar  task.  Signals 
were  noise  bands,  nominally  200  Hz  wide,  with  center  frequencies 
(CFs)  of  0.9,  1.8,  3.6,  and  7.2  kHz.  The  signal  was  presented  in 
a  background  of  white  noise. 

Subjects  viewed  a  modified  waterfall-type  display  and/or 
heard  the  same  signals  presented  aurally.  Independent  variables 
included  (1)  display  modality  (visual,  aural,  or  bimodal) ,  (2) 

signal  uncertainty  (0  or  2  bits),  and  (3)  lateralized  (dichotic) 
or  diotic  presentation  of  the  auditory  signal. 

The  auditory  stimulus  (signal  plus  masker)  was  lateralized 
in  one  condition  by  low-passing  the  left  ear  input  and  high- 
passing  the  right  ear  input.  This  caused  the  low  frequency  signal 
to  be  heard  on  the  left  side  of  the  head,  moderate  frequency 
signals  in  the  middle,  and  the  highest  frequency  signal  on  the 
right.  The  visual  display  also  presented  the  stimulus  spectrum 
horizontally,  with  lowest  frequencies  on  the  left.  Therefore, 
when  the  display  was  bimodal  and  the  auditory  stimulus  was 
lateralized,  the  positions  of  the  signal  on  the  visual  and 
auditory  displays  were  correlated.  This  was  called  the  "high 
compatibility"  condition.  When  the  auditory  signal  was  not 
lateralized,  there  was  no  spatial  correlation  between  the  visual 
and  auditory  signals?  this  was  called  the  "low  compatibility" 
condition. 

Signal  uncertainty  was  manipulated  by  presenting  either  a 
single  signal  repeatedly  on  a  given  block  of  test  trials  (0  bits 
uncertainty) ,  or  by  presenting  one  of  four  signals  (2  bits)  in  a 
random  sequence. 
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Each  test  trial  involved  a  two-alternative  forced  choice. 
Signal  level  was  adaptively  varied  within  each  block  of  test 
trials  to  maintain  a  fixed  probability  of  correct  response.  The 
sigrial-to-noise  ratio  (SNR)  required  for  detection  was  estimated 
by  averaging  the  maximum  and  minimum  signal  levels  in  the  latter 
portion  of  each  test  block. 

Stimuli s  Signals  were  0  to  100  Hz  bands  of  noise  multiplied 
by  sinusoids  with  frequencies  of  0.9,  1.8,  3.6,  or  7.2  kHz.  The  3 
dB  bandwidth  of  each  signal  was  209  Hz,  and  the  rolloff  was  115 
dB  per  octave  on  both  sides.  The  signals  were  amplitude 
modulated  by  a  15  Hz  sine  wave  with  an  RMS  amplitude  of  1.5  V  ac 
and  a  5  V  dc  offset.  The  signals  had  20  ms  rise  and  fall  times 
which  followed  a  cosine-squared  function.  The  spectrum  level  of 
the  noise  masker  was  constant  at  25  dB  (re:  .0002  dynes/cm2)  out 
to  14  kHz,  after  which  it  gradually  rolled  off.  The  headphones 
were  Sennheiser  model  430. 

The  same  input  was  fed  to  both  the  visual  and  the  auditory 
displays,  except  that  the  auditory  input  was  further  filtered  in 
the  lateralized  condition.  Lateralization  was  accomplished  by 
passing  the  left  channel  through  an  equalizer  in  series  with  a 
low-pass  filter,  and  the  right  channel  through  an  equalizer  in 
series  with  a  high-pass  filter.  The  transfer  functions  for  each 
channel,  as  plotted  directly  from  a  1024-point  digital  spectrum 
analyzer,  are  shown  in  Figure  1.  The  solid  horizontal  line  shows 
the  level  of  the  diotic  input.  Note  that  the  high-pass  (right 
ear)  channel  was  boosted  about  12  dB  at  7.2  kHz  relative  to  the 
diotic  input.  Similarly,  the  low-pass  (left  ear)  channel  was 
boosted  12  dB  at  900  Hz.  The  equalizer  and  filter  settings  were 
developed  in  pilot  testing  with  four  listeners  who  had 
bilaterally  normal  hearing.  The  filter  characteristics  and  signal 
CFs  were  chosen  so  that  the  perceived  positions  of  the  signals 
were  maximally  separated  and  reliably  judged. 


Visual  signals  were  presented  on  a  Spectral  Dynamics 
waterfall-type  display,  which  was  part  of  a  sonar  system.  The 
input  was  fed  directly  to  the  system  display  board,  bypassing 
filters.  Only  the  portion  of  the  input  from  0  to  10  kHz  was 
presented  on  the  visual  display.  The  horizontal  width  of  the 
signal  plus  noise  on  the  display  was  25  cm,  which  included  400 
pixels.  The  average  luminance  of  the  noise  signal  alone  was 
0.026  cd/m2,  and  the  average  luminance  measured  in  the  center  of 
the  signals  at  an  SNR  of  0  dB  was  0.078  cd/m2.  The  total  vertical 
height  of  the  10  raster  lines  used  was  1.2  cm.  The  viewing 
distance  was  85  cm. 

Each  raster  line  on  the  visual  display  represented  the 
stimulus  energy  integrated  over  a  period  of  0.1  s .  The  most 
recent  power  spectrum  was  displayed  on  the  top  raster  line,  and 
the  earlier  spectra  were  each  moved  down  one  line  each  0.1  s. 
Black  paper  was  used  to  cover  the  lower  part  of  display,  allowing 
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ATTENUATION  MB) 


Figure  1.  Amplitude  responses  of  filters  used  to  lateralize  the 


only  10  raster  lines  (1.0  s  of  signal)  to  be  viewed. 


Procedure :  Subjects  were  tested  individually  in  a  partially 
darkened,  double-walled  audiometric  booth.  Each  subject 
participated  in  three  training  sessions  and  ten  test  sessions  of 
approximately  90  minutes  duration  each. 

An  adaptive  tracking  procedure  was  used  to  estimate  the 
signal  level  required  for  detection  in  each  condition.  Each 
estimate  was  based  on  a  block  of  80  test  trials.  During  the  test 
block,  the  signal  level  was  decreased  a  predetermined  step  size 
after  every  three  consecutive  correct  responses,  and  increased 
the  same  step  size  after  each  error.  This  constrained  the 
percentage  of  correct  responses  to  be  79.4  (or  equivalently, 
d'=l .  16)  .  The  signal  level  in  each  test  block  was  initially  set 
10  dB  higher  than  an  initial  level  based  on  previous  testing.  The 
step  size  was  6  dB  until  the  first  error,  and  2  dB  for  the 
remainder  of  the  block. 

On  each  test  trial,  the  subject  judged  whether  the  signal 
had  appeared  in  the  first  or  second  of  two  observation  intervals 
by  pressing  one  of  two  4-inch  square  boxes  arranged  horizontally 
on  touch  screen,  positioned  within  easy  reach  of  the  right  index 
finger.  The  observation  intervals  were  1.0  s  long,  and  were 
separated  by  a  0.5  s  pause.  The  system  waited  4.0  s  for  the 
subject's  response,  after  which  a  prompting  message  appeared  on 
the  touch  screen  and  the  touch  screen  monitor  beeped.  The  system 
then  waited  up  to  4.0  additional  seconds  for  a  response  before 
delivering  feedback.  If  the  response  was  correct,  a  "+"  sign  was 
displayed  and  one  short  beep  was  sounded;  feedback  for  errors  and 
non-responses  was  a  sign  and  two  beeps.  The  next  test  trial 

began  one  second  after  the  feedback. 

Over  the  three  training  sessions,  each  subject  completed  25 
blocks  of  practice  trials,  representing  each  of  the  25  stimulus 
conditions.  The  conditions  were  run  in  a  fixed  order  in  training. 
In  each  test  session,  each  subject  completed  nine  test  blocks, 
including  three  in  the  auditory  modality  only,  three  visual  only, 
and  three  bimodal.  The  auditory  and  visual  conditions  were 
alternated  over  the  first  six  blocks,  with  an  auditory  or  visual 
block  randomly  selected  to  be  the  starting  condition.  The  same 
signal  or  signals  were  used  throughout  a  given  session.  Each  of 
the  four  signals  was  used  once  in  sessions  1  through  4,  the  order 
being  random.  In  session  5,  all  four  signals  were  used  (high 
uncertainty  condition) .  In  the  first  five  sessions,  the  auditory 
display  was  lateralized  for  three  of  the  subjects  (determined 
randomly)  and  diotic  for  the  others.  Sessions  6  through  10 
replicated  the  first  five  sessions,  except  that  the  auditory 
display  was  switched  to  the  type,  lateralized  or  diotic,  that  the 
subject  had  not  experienced  in  sessions  1  through  5. 

Running  the  single-modality  test  blocks  prior  to  the  bimodal 
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blocks  in  each  session  made  it  possible  to  equate  the  auditory 
and  visual  signals  for  equal  detectability  in  the  subsequent 
bimodal  test  blocks.  The  initial  level  of  the  auditory  signal  in 
the  bimodal  blocks  was  set  at  the  average  measured  "threshold" 
from  the  three  auditory  test  blocks  run  earlier  in  the  same 
session.  The  initial  level  for  the  visual  signal  was  set  in  the 
same  manner.  The  signal  levels  were  set  individually  for  each 
subject,  based  on  his  or  her  prior  data. 

In  the  high  uncertainty  sessions  (numbers  5  and  10) ,  the 
signal  levels  for  the  single-modality  test  blocks  were  set  at  the 
average  measured  threshold  from  the  low  uncertainty,  single¬ 
modality  test  blocks  run  in  the  four  preceding  sessions.  The 
initial  signal  levels  for  the  low  uncertainty,  single  modality 
blocks  in  sessions  1  to  4  and  6  to  9  were  set  at  each  individual 
subject's  measured  threshold  in  training  for  the  same  condition. 
Initial  thresholds  in  training  blocks  were  set  to  the  same  level 
for  all  subjects,  based  on  pilot  data. 

At  the  start  of  each  test  block,  the  subject  was  told  the 
display  modality,  the  number  of  alternative  signals  which  could 
appear,  and  whether  the  auditory  signal  would  be  lateralized. 

Care  was  also  taken  to  ensure  that  the  subject  had  the  earphones 
oriented  properly. 

Subjects;  The  subjects  were  two  women  and  three  men  between 
the  ages  of  22  and  32  years.  Subjects  were  audiometrically  normal 
and  had  normal  vision.  Hearing  was  tested  to  ANSI  S3. 21-1978 
specifications,  and  vision  was  low  normal  or  better  on  the 
Vistech,  Inc.  VCTS  6000  contrast  sensitivity  test. 

Data  Reduction;  The  raw  data  for  each  block  of  test  trials 
consisted  of  a  trial-by-trial  record  of  attenuation  relative  to 
the  initial  level  at  the  beginning  of  the  block.  The  record 
showed  a  local  maximum  or  peak  whenever  the  subject  made  an  error 
after  a  series  of  three  consecutive  correct  responses,  which 
caused  the  signal  attenuation  to  be  decreased  on  the  next  trial. 
It  showed  a  local  minimum  or  valley  whenever  three  consecutive 
correct  responses  were  made  after  one  or  more  errors,  which 
caused  the  attenuation  to  be  increased  for  the  next  trail. 

For  each  block,  the  average  deviation  from  the  initial 
attenuation  was  computed  by  taking  the  mean  of  an  equal  number  of 
peaks  and  valleys  after  the  first  peak  that  occurred  after  a 
valley.  If  the  number  of  peaks  and  valleys  after  the  first 
valley  and  peak  were  not  equal,  one  peak  or  valley  closest  in 
time  to  the  beginning  of  the  block  was  dropped.  The  deviation  was 
added  to  the  initial  attenuation  and  converted  to  the  signal  to 
noise  ratio  (SNR) .  In  addition,  the  data  from  each  block  were 
fit  with  a  psychometric  function  of  the  form; 

d'  =  m  (E/N0)k 
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RESULTS 


Effects  of  Signal  Center  Frequency  and  Signal  Type  in  Sinale- 
Modality  Displays:  Figure  2  shows  the  SNR  required  for  detection 
as  a  function  of  the  CF  of  the  signal  and  type  of  signal  (visual, 
diotic  auditory,  and  dichotic  auditory) .  These  data  are  for  the 
low  uncertainty  condition  only.  Up  to  and  including  the  3.6  CF 
kHz  signal,  the  visual  and  auditory  SNRs  required  for  detection 
were  roughly  comparable.  However,  for  the  7.2  kHz  signal, 
auditory  SNRs  were  dramatically  higher.  Analysis  of  variance 
(ANOVA)  showed  that  the  effects  of  CF  and  the  interaction  of  CF  X 
Signal  Type  were  both  significant  at  the  p=.001  level 
(F (3 , 12) =11 . 3  and  £(6, 24) =11. 7 ,  respectively).  The  main  effect 
of  Signal  Type  was  significant  at  the  p=.05  level  (F(2,8)=5.8) . 

In  addition  to  the  difference  between  visual  and  auditory 
signals  noted  above,  it  appears  from  Figure  2  that  there  is  a 
difference  in  SNR  for  7.2  kHz  CF  signal  for  diotic  as  opposed  to 
dichotic  auditory  signal.  This  difference  was  not  significant  by 
Tukey's  Honestly  Significant  Difference  test  (g(12,48)=1.8, 
p>. 05) . 

Effects  of  Signal  Uncertainty  and  Signal  Type  in  Single-Modality 
Displays t  Figure  3  shows  the  data  for  single-modality  displays 
averaged  over  CFs.  The  SNR  required  to  detect  the  signal 
increased  significantly  with  signal  uncertainty  (F(l, 4) =15. 8,  p< 
.025).  Signal  uncertainty  had  a  much  greater  effect  on  visual 
signals  than  on  auditory  signals  (F (2 , 8) =24 . 7 ,  g<  .001).  In  the 
low  uncertainty  condition,  SNR  for  visual  signals  averaged  0.8  dB 
lower  than  that  for  auditory  signals  presented  diotically.  In 
the  higher  uncertainty  condition  however,  SNR  for  visual  signals 
averaged  1.3  dB  greater  than  that  for  auditory  signals  presented 
diotically.  Mean  SNRs  for  auditory  signals  presented  dichotically 
tended  to  be  greater  than  those  for  both  visual  and  diotic 
auditory  signals  at  both  levels  of  uncertainty.  However,  the 
overall  main  effect  of  signal  Type  in  Figure  3  was  not 
significant  (F(2,8)=3.2,  e>  .05). 

Effects  of  Bimodal  versus  Single-Modality  Presentation:  Table  1 
shows  the  effect  of  bimodal  as  opposed  to  single-modality 
presentation.  The  values  in  this  table  are  the  differences  in 
SNRs  between  bimodal  and  single-modality  displays  for  each  of  the 
conditions  shown.  A  2  X  2  ANOVA  was  performed  to  examine 
differences  among  the  four  means  at  the  bottom  of  the  table 
(i.e.,  collapsing  over  signal  CF) . 

There  was  a  highly  significant  overall  advantage  of  bimodal 
as  opposed  to  single-modality  displays.  For  bimodal  displays  the 
SNR  required  to  detect  the  signal  averaged  1.1  dB  less  than  the 
SNR  for  single-modality  displays  (£(1,19)=42.4,  p<  .001). 
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Table  1. 


Bimodal  facilitation  as  a  function  of  center  frequency 
of  signal,  type  of  authority  display,  and  signal 
uncertainty. 


Signal 

uncer¬ 

tainty 

Center 

frequency 

of 

signal 

(kHz) 

Tvoe  of  auditorv  disolav  1 

Diotic 

Lateralized 

(Low  compatibility) 

(High  compatibility) 

0.9 

-1.1 

-1.8 

1.8 

-1.0 

-1.3 

Low 

3.6 

-0.5 

-0.9 

7.2 

-0.8 

-0.9 

Means  for 

.ow  uncer- 

-0.8 

-1.2 

taintv  condition 

Hiah  1  - 

-1.5 

-0.9 

Note:  Entries  are  10  log  S/N  for  bimodal  displays  minus  the  same 

value  for  the  corresponding  unimodal  displays.  Negative  numbers 
indicate  greater  sensitivity. 


Contrary  to  expectations,  the  bimodal  facilitation  did  not 
increase  significantly  with  signal  uncertainty  (F( 1,4) =0.8,  p> 
.05).  Increased  display  Compatibility  (diotic  versus  dichotic 
auditory  signal)  also  failed  to  produce  greater  bimodal 
facilitation  (F(l,4)=0.2,  p>  .05).  The  interaction  of 
Compatibility  X  Uncertainty  was  also  not  significant  (F(l,4)=3.6, 
E>  • 05 ) . 

An  additional  ANOVA  was  performed  to  determine  whether  the 
advantage  of  bimodal  presentation  changed  with  signal  CF  in  the 
low  signal  Uncertainty  condition.  These  data  are  shown  in  the 
upper  four  rows  of  Table  1.  The  effect  of  signal  CF  was  not 
significant  (F(3,12)=l.l,  p>  .05).  The  effect  of  Compatibility 
(diotic  versus  dichotic  auditory  stimulus)  was  also  not 
significant  (F (1,4) =3.0,  p>  .05) .  Nor  was  the  interaction  of  CF 
X  Compatibility  significant  in  these  data  F(3,12)=0.2,  p>  .05). 

Practice  Effects  within  Sessions:  In  each  test  session,  test 
blocks  in  the  visual  and  auditory  display  conditions  were  run 
prior  to  the  bimodal  condition  so  that  results  of  the  former 
could  be  used  to  equate  the  detectability  of  visual  and  auditory 
signals  in  the  latter.  Even  though  the  subjects  were  well 
practiced  before  the  test  sessions  began,  it  was  possible  that 
practice  effects  within  test  sessions  might  emphasize  any 
difference  in  SNR  between  bimodal  and  single-modality  test 
blocks.  That  is,  any  within-session  practice  effect  could  lead  to 
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overestimation  of  the  bimodal  effect. 

'The  wittiin-session  practice  effect  was  therefore  evaluated . 
Since  each  Stimulus  condition  was  tested  three  times  in  each 
session,  it  was  possible  to  examine  trends  over  repeated  test 
blocks  of  the  same  stimulus  conditions*  The  mean  SNRs  required 
for  detection  were  computed  for  first,  second,  and  third 
repetitions  of  test  conditions,  averaqed  over  all  subjects  and 
test  conditions,  'fhe  mean  snrs  were  -2*1,  *-2.2,  and  -2*o  dB  for 
first/,  second,  and  third  repetitions,  respectively*  For  four  of 
the  five  subjects,  the  mean  SNR  required  for  detection  increased 
from  the  first  to  the  third  repetition.  These  results  show  that 
there  was  no  practice  effect;  that  is,  the  change  in  SNR  was 
small  and  in  the  wrong  direction. 

Table  2  shows  the  average  Values  of  k  when  the  data  Were  fit 
with  the  function: 

d'=  m  (E/N0)k* 

The  average  value  of  k  across  conditions  is  1*54,  With  generally 
little  variation  among  conditions.  In  particular,  the  similarity 
of  the  k  values  for  the  auditory  and  Visual  conditions  means  that 
when  performance  was  equated  at  one  level,  it  Was  also  equated  as 
signal  level  varied  in  the  bimodal  condition. 


Table  2.  Average  Value  Of  k  from  maximum  likelihood  fit  to  the 
psychometric  function,  d'  =  m 


signal 

uncer¬ 

tainty 

Means 

Center 

frequency 

Of 

signal 

fkHzl 

Auditory  .  |  Bimodal 

Tvoe  of  Auditorv  Disolav 

Visual 

Lateral 

Diotic 

(HC) 

(LC) 

Mj 

0.9 

1.39 

1.23 

1*52 

1.43 

1.67 

1.45 

1.8 

1.27 

1.72 

1.75 . 

1.68 

1.52 

1.59 

3.6 

1.41 

1.82 

1.72 

1.31 

1.44 

1.54 

7.2 

1.61 

1*86 

1*58 

.  1.45 

2*17 

1.73 

Means  . 

w 

1.66  . 

.  1*64  ... 

1.47 

1.70 

1.58 

j-- 

1.13 

-  1*67 

1.65 

1.39 

1.50 

Overa] 

LI  Means  - 

1.53 

1.40 

1.66 

1.56 

1.55 

1.54 

High  compatibility  condition 
Low  compatibility  condition 
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DISCUSSION 


The  results  show  a  highly  consistent  advantage  of  bimodal 
over  single-modality  displays.  The  SNR  required  to  detect  the 
signal  averaged  1.1  dB  less  with  bimodal  displays.  This  finding 
is  in  close  agreement  with  the  two  previous  applied  studies  which 
used  detection  rather  than  vigilance  or  search  tasks  (Luria  and 
Jacobsen,  1986;  Lewandowski  and  Kobus,  in  press).  Both  of  these 
studies  reported  bimodal  facilitations  of  about  1.0  dB.  The  fact 
that  the  effect  was  replicated  in  the  present  study,  which  used  a 
TSD-based  method  and  equated  the  signals  for  detectability, 
provides  strong  evidence  that  bimodal  displays  enhance  operators' 
perceptual  sensitivity;  that  is,  the  effect  is  not  due  to  changes 
in  response  criterion,  nor  a  statistical  artifact. 

The  bimodal  improvement  is  in  good  agreement  with  that 
predicted  by  optimal  integration  of  the  information  in  the  two 
channels  (Green  and  Swets,  1966,  p.  238).  Assuming  that  the 
information  in  the  two  channels  is  independent,  the  bimodal  d' 
should  be  the  square  root  of  2.0  times  the  unimodal  d's.  Since 
the  overall  psychometric  function  has  an  exponent  of  1.54,  a 
square  root  of  2.0  change  in  performance  is  comparable  to  a 
threshold  decrease  of  0.96  dB.  Thus,  the  results  are  consistent 
with  optimal  integration  of  the  information  in  the  two 
modalities. 

Both  the  present  research  and  the  Luria  &  Jacobsen  and 
Lewandowski  &  Kobus  studies  used  sonar-type  displays  and  stimuli. 
But  bimodal  displays  are  potentially  applicable  to  a  wide  variety 
of  human-machine  systems,  including  aircraft  displays  and  air- 
traffic  control.  A  useful  direction  for  future  research  would  be 
to  investigate  the  generality  of  the  bimodal  effect  with  a  wider 
range  of  displays,  signals,  and  tasks. 

In  addition  to  confirming  the  benefit  of  bimodal 
presentation,  the  present  results  provide  information  on  how  task 
and  display-design  variables  affect  detection  performance.  As 
expected,  increased  signal  uncertainty  produced  a  decrement  in 
detection  performance.  For  single-modality  displays,  increasing 
uncertainty  from  0  to  2  bits  increased  the  SNR  required  to  detect 
the  signal  by  an  average  of  1.8  dB.  The  uncertainty  decrement 
was  significantly  greater  for  visual  than  for  auditory  displays. 
Increased  signal  uncertainty  increased  the  SNR  for  visual 
detection  an  average  of  3.0  dB,  but  increased  the  SNR  for 
auditory  detection  only  1.2  dB.  For  purposes  of  comparison,  the 
uncertainty  decrement  was  computed  for  an  ideal  processor 
monitoring  one  versus  four  independent  channels,  using  the  method 
described  by  Nolte  and  Jaarsma  (1967)  and  Peterson,  Birdsall,  and 
Fox  (1954).  This  result  was  also  1.2  dB,  suggesting  that  the 
auditory  display  allowed  subjects  to  monitor  the  four  channels  in 
parallel.  These  findings  suggest  that  auditory  displays  may 
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offer  practical  advantages  for  real-world  sonar  operations,  where 
signal  uncertainty  is  often  high. 

The  advantage  of  bimodal  as  opposed  to  single-modality 
displays  did  not  increase  with  signal  uncertainty.  Of  course, 
the  fact  that  these  variables  had  additive  effects  on  the  SNR 
required  for  detection  does  not  imply  that  their  effects  would  be 
additive  in  terms  of  the  probability  of  a  correct  response,  P(C) . 
Since  the  relationship  between  SNR  and  P(C)  is  non-linear  in 
general,  variables  whose  effects  are  additive  in  terms  of  SNR  may 
be  non-additive  if  studied  by  a  procedure  that  holds  SNR  constant 
and  allows  P(C)  to  vary. 

Contrary  to  expectation,  increased  compatibility  between  the 
auditory  and  visual  displays  (going  from  diotic  to  dichotic 
auditory  stimuli)  did  not  produce  greater  bimodal  facilitation. 
Dichotic  auditory  stimuli  also  produced  poorer  detection 
performance  than  did  diotic  stimuli  in  single-modality  displays. 
Introspective  reports  from  the  subjects  indicated  that  the 
dichotic  (lateralized)  auditory  stimuli  did  not  provide  strong 
spatial  cues.  Although  the  lowest  frequency  signal  (0.9  kHz  CF) 
was  clearly  heard  on  the  left  side,  the  perceived  locations  of 
the  higher  frequency  signals  were  somewhat  variable  and 
uncertain.  It  is  concluded  that  lateralizing  the  auditory 
stimulus  by  varying  the  interaural  amplitude  alone  is  not  a 
useful  way  to  increase  display  compatibility,  and  thereby 
detection  performance.  This  does  not  rule  out  the  possibility 
that  other  means  of  enhancing  compatibility  would  facilitate 
detection.  In  particular,  it  is  recommended  that  future  research 
use  interaural  phase  differences  and/or  mimic  the  direction- 
dependent  amplitude  response  of  the  human  head  and  pinna. 


< 
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on  the  bimodal  display.  It  was  expected  that  spatially  correlated  auditory  signals 
would  facilitate  detection  in  the  bimodal  condition  by  cluing  the  operator  where  to 
look  on  the  visual  display. 
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19.  Cont'd. 

Increased  signal  uncertainty  produced  an  increase  in  the  average  signal-to-noise 
ratio  (SNR)  at  which  the  signal  could  first  be  detected.  The  increase  in  the  SNR 
needed  for  detection  was  significantly  greater  for  the  visual  display  (3.0  dB)  than 
for  the  auditory  display  (1.2  dB) .  On  the  bimodal  displays,  signals  were  detected 
at  significantly  lower  SNRs  than  on  single-modality  displays  (average  1.1  dB 
difference) .  Increased  compatibility  between  the  visual  and  auditory  signals  did  not 
increase  the  advantage  of  bimodal  displays;  nor  did  the  advantage  of  bimodal  displays 
change  with  signal  uncertainty. 

The  findings  show  that  bimodal  displays  improve  signal  detection  performance  in 
!sonar  systems.  They  rule  out  the  possibility  that  the  advantage  of  bimodal  displays 
is  attributable  to  changes  in  operators1  response  criteria  and/or  artifacts  caused  by 
differences  in  the  detectability  of  the  visual  and  auditory  signals.  They  also 
suggest  that  auditory  displays  offer  advantages  for  real-world  sonar  operations,  where 
signal  uncertainty  is  often  high.  The  method  used  to  make  auditory  and  visual  signals 
spatially  compatible  in  this  study  was  not  successful.  More  effective  methods  for 
creating  spatial  compatibility  are  recommended  for  future  research. 
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